fix(worker): honor explicit worker connection schemes#1485
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds case-insensitive scheme parsing, routes health checks by scheme (http vs grpc), honors explicit URL schemes in connection detection, accepts ChangesgRPC over TLS Support
Sequence DiagramsequenceDiagram
participant Client as Caller
participant Detect as DetectConnectionModeStep
participant Util as util helpers
participant HTTP as HTTP_health_check
participant GRPC as gRPC_health_check
Client->>Detect: execute(url)
Detect->>Util: explicit_connection_mode(url)
alt explicit HTTP
Detect->>HTTP: http_health_url -> probe /health
HTTP-->>Detect: success/fail
else explicit gRPC
Detect->>GRPC: grpc_reachable_url -> run gRPC health checks
GRPC-->>Detect: success/fail
else none (bare host:port)
Detect->>HTTP: probe
Detect->>GRPC: probe
end
Detect-->>Client: set connection_mode or error
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request introduces explicit URL scheme detection in the connection detection workflow, allowing the system to honor grpc://, http://, and https:// prefixes. When a scheme is present, the system performs a targeted health check instead of probing both protocols in parallel. Feedback suggests replacing the manual string prefix matching with a proper URL parsing library to improve reliability, handle case sensitivity more robustly, and avoid potential issues with relative URLs.
There was a problem hiding this comment.
Clean, well-structured change. The explicit scheme detection is a good UX improvement over always probing both protocols. Tests cover the key cases.
Two minor nits posted inline (both 🟡):
- Case normalization mismatch:
to_ascii_lowercase()in the detector accepts mixed-case schemes that downstream functions can't handle — suggest matching exact lowercase prefixes instead. - Redundant protocol variable: error message includes both
{protocol}and{connection_mode}Display — one suffices.
CatherineSue
left a comment
There was a problem hiding this comment.
The fix LGTM.
I think it would be also better to make try_http_reachable and try_grpc_reachable more robust. The bug exists because they both try to use strip_protocol. Maybe they both need to be more strict. Such as:
try_http_reachable:
accepts http://, https://, or bare host:port
rejects grpc://, grpcs://
try_grpc_reachable:
accepts grpc://, grpcs://, or bare host:port
rejects http://, https://
fddb01a to
8aabf5c
Compare
8aabf5c to
c652b76
Compare
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/grpc_client/src/mlx_engine.rs`:
- Around line 125-131: Add a new async Tokio test named
test_grpcs_scheme_conversion next to the existing
test_client_connect_invalid_endpoint to assert the new scheme conversion; call
MlxEngineClient::connect("grpcs://localhost:50051").await and assert the result
is Err (connection failure expected) so the failure is due to connection and not
scheme parsing, ensuring the grpcs:// → https:// path in the endpoint
normalization branch is covered.
- Around line 125-131: The scheme-prefix checks for building http_endpoint in
MlxEngineClient::connect() (and connect_with_trace_injector()) are
case-sensitive; make them case-insensitive by lowercasing a temporary copy of
endpoint (e.g., let lower = endpoint.to_lowercase()) and use
lower.strip_prefix("grpc://") / lower.strip_prefix("grpcs://") to detect the
scheme, but when constructing the replacement URL use the substring of the
original endpoint after the matched prefix length (so host/port/case are
preserved) to build "http://{rest}" or "https://{rest}"; apply this change in
the code that sets http_endpoint so user-supplied schemes like GRPC:// or
GrPcS:// are handled correctly.
In `@crates/grpc_client/src/sglang_scheduler.rs`:
- Around line 146-152: Add a unit test that mirrors
test_client_connect_invalid_endpoint() to cover the new grpcs:// scheme: call
the same code path that computes http_endpoint (the logic using
endpoint.strip_prefix in sglang_scheduler.rs) with a "grpcs://host:port" input
and assert the resulting http_endpoint is "https://host:port"; ensure the test
lives in the same test module as test_client_connect_invalid_endpoint() so it
exercises the same conversion logic and fails if the grpcs:// -> https://
mapping regresses.
- Around line 146-152: The scheme check in sglang_scheduler.rs uses
case-sensitive strip_prefix on endpoint, so endpoints like "GRPC://" or
"GRPCS://" are not recognized; update the logic around http_endpoint to do
case-insensitive matching by comparing a lowercase copy (e.g., let lower =
endpoint.to_ascii_lowercase()) and then if lower.strip_prefix("grpc://") /
"grpcs://", derive the addr by slicing the original endpoint past the matched
prefix (so original casing of the host is preserved) and format into
"http://{addr}" or "https://{addr}" accordingly; update the block that sets
http_endpoint to use this case-insensitive check on endpoint while still using
the original endpoint string for the formatted address.
In `@crates/grpc_client/src/trtllm_service.rs`:
- Around line 145-151: Add a unit test that exercises the scheme conversion
logic around endpoint.strip_prefix so grpcs:// URIs are converted to https://;
specifically, create a test that supplies an endpoint like
"grpcs://example.com:1234" and asserts the resulting http_endpoint equals
"https://example.com:1234" (similar to existing tests for grpc:// → http://).
Place the test adjacent to other tests for the module and reference the same
conversion code that computes http_endpoint (the block using
endpoint.strip_prefix and format!).
- Around line 145-151: The scheme matching for endpoint is case-sensitive and
misses mixed-case schemes like "GRPCS://"; update the logic in the connect flow
(where endpoint and http_endpoint are computed, e.g. GrpcClient::connect in
trtllm_service.rs) to perform case-insensitive matching by using
endpoint.to_ascii_lowercase() for the prefix checks and then slicing the
original endpoint to get the addr (so the host/port retains original casing).
Concretely: compute a lower = endpoint.to_ascii_lowercase(), check
lower.strip_prefix("grpc://") / lower.strip_prefix("grpcs://"), and when a match
is found use the corresponding byte/char index to take the addr from the
original endpoint and format "http://{addr}" or "https://{addr}" into
http_endpoint.
In `@crates/grpc_client/src/vllm_engine.rs`:
- Around line 146-152: The new scheme conversion in vllm_engine.rs (the endpoint
normalization that switches "grpc://"→"http://" and "grpcs://"→"https://" inside
the block that builds http_endpoint) lacks a unit test; add a test (e.g.,
test_endpoint_scheme_conversion or test_grpcs_to_https_conversion) that calls
the function or constructs the same normalization path with an input
"grpcs://example.com:1234" and asserts the resulting http_endpoint equals
"https://example.com:1234"; place the test near other vllm_engine tests,
exercise both "grpc://" and "grpcs://" cases, and run cargo test to ensure
coverage.
- Around line 146-152: The scheme check for building http_endpoint is
case-sensitive and misses mixed-case prefixes; fix by computing a lowercase copy
(e.g., let lower = endpoint.to_ascii_lowercase()) and test strip_prefix against
"grpc://" and "grpcs://" on that lowercase string, then when a match is found
derive the addr by slicing the original endpoint string past the matched prefix
length (so you preserve the original rest of the URL) and format to
"http://{addr}" or "https://{addr}"; update the logic around http_endpoint and
endpoint.strip_prefix to use this lowercased check and original slicing.
In `@model_gateway/src/workflow/steps/local/detect_connection.rs`:
- Around line 19-27: The explicit_connection_mode function currently uses
case-sensitive starts_with checks and should detect schemes case-insensitively
to match util.rs behavior; update explicit_connection_mode to compare the URL
scheme using eq_ignore_ascii_case (or lowercase the prefix) for "grpc", "grpcs",
"http", and "https" so mixed-case schemes like "GRPC://..." are recognized as
ConnectionMode::Grpc/Http, and add a unit test for a mixed-case URL (e.g.,
"GRPC://localhost:30001") to cover this case.
🪄 Autofix (Beta)
✅ Autofix completed
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3043b542-5812-4560-8b71-b1c1d1c00b40
📒 Files selected for processing (6)
crates/grpc_client/src/mlx_engine.rscrates/grpc_client/src/sglang_scheduler.rscrates/grpc_client/src/trtllm_service.rscrates/grpc_client/src/vllm_engine.rsmodel_gateway/src/workflow/steps/local/detect_connection.rsmodel_gateway/src/workflow/steps/util.rs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c652b76f73
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. Fixes Applied SuccessfullyFixed 4 file(s) based on 4 unresolved review comments. Files modified:
Commit: The changes have been pushed to the Time taken: |
There was a problem hiding this comment.
♻️ Duplicate comments (4)
crates/grpc_client/src/mlx_engine.rs (1)
125-131:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winImplement case-insensitive scheme matching.
The scheme checks use case-sensitive
strip_prefixand won't handle mixed-case schemes likeGRPC://orGrPcS://. Past review analysis showsworker.rs:952passesmetadata.spec.urldirectly toGrpcClient::connect(), and sincegrpc_base_urlinutil.rspreserves the original URL casing when a scheme is detected, uppercase schemes will bypass the conversion and cause connection failures.🔧 Proposed fix
- let http_endpoint = if let Some(addr) = endpoint.strip_prefix("grpc://") { + let lower = endpoint.to_ascii_lowercase(); + let http_endpoint = if let Some(_) = lower.strip_prefix("grpc://") { + let addr = &endpoint[7..]; format!("http://{addr}") - } else if let Some(addr) = endpoint.strip_prefix("grpcs://") { + } else if let Some(_) = lower.strip_prefix("grpcs://") { + let addr = &endpoint[8..]; format!("https://{addr}") } else { endpoint.to_string() };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/mlx_engine.rs` around lines 125 - 131, The scheme-matching for building http_endpoint in the mlx_engine.rs block (the if/else that currently uses endpoint.strip_prefix("grpc://") and strip_prefix("grpcs://")) is case-sensitive and fails for mixed/upper-case schemes; update this block to perform case-insensitive detection (e.g. compare a lowercased view of endpoint or use a case-insensitive starts-with) and then strip only the scheme portion while preserving the original remainder of endpoint so GrpcClient::connect() (and grpc_base_url in util.rs) receive the correct host/path; adjust the logic around the http_endpoint variable and the endpoint handling to use these case-insensitive checks.crates/grpc_client/src/vllm_engine.rs (1)
145-152:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winImplement case-insensitive scheme matching.
The scheme checks use case-sensitive
strip_prefixand won't handle mixed-case schemes likeGRPC://orGrPcS://. Past review analysis shows that direct callers bypass workflow layer normalization, and sincegrpc_base_urlinutil.rspreserves the original URL casing when a scheme is detected, mixed-case schemes will fail the conversion and cause connection failures.🔧 Proposed fix
// Convert gRPC schemes to tonic-compatible HTTP(S) endpoints. - let http_endpoint = if let Some(addr) = endpoint.strip_prefix("grpc://") { + let lower = endpoint.to_ascii_lowercase(); + let http_endpoint = if let Some(_) = lower.strip_prefix("grpc://") { + let addr = &endpoint[7..]; format!("http://{addr}") - } else if let Some(addr) = endpoint.strip_prefix("grpcs://") { + } else if let Some(_) = lower.strip_prefix("grpcs://") { + let addr = &endpoint[8..]; format!("https://{addr}") } else { endpoint.to_string() };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/vllm_engine.rs` around lines 145 - 152, The current scheme matching on `endpoint` uses case-sensitive `strip_prefix` and fails for mixed-case schemes; update the conversion that sets `http_endpoint` so it matches schemes case-insensitively by comparing a lowercased view (e.g., `let lower = endpoint.to_lowercase()`) and, when a match is found, take the remainder from the original `endpoint` by slicing with the scheme length (so casing of the address is preserved) and then format to "http://{addr}" or "https://{addr}". Make this change in the block that computes `http_endpoint` (the code referencing `endpoint` and producing `http_endpoint`) so both "grpc://" and "grpcs://" are handled regardless of case.crates/grpc_client/src/sglang_scheduler.rs (1)
145-152:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winImplement case-insensitive scheme matching.
The scheme checks use case-sensitive
strip_prefixand won't handle mixed-case schemes likeGRPC://orGrPcS://. Past review analysis confirms thatworker.rs:952and other callers pass URLs directly without normalization, and mixed-case schemes will bypass the conversion logic causing connection failures.🔧 Proposed fix
// Convert gRPC schemes to tonic-compatible HTTP(S) endpoints. - let http_endpoint = if let Some(addr) = endpoint.strip_prefix("grpc://") { + let lower = endpoint.to_ascii_lowercase(); + let http_endpoint = if let Some(_) = lower.strip_prefix("grpc://") { + let addr = &endpoint[7..]; format!("http://{addr}") - } else if let Some(addr) = endpoint.strip_prefix("grpcs://") { + } else if let Some(_) = lower.strip_prefix("grpcs://") { + let addr = &endpoint[8..]; format!("https://{addr}") } else { endpoint.to_string() };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/sglang_scheduler.rs` around lines 145 - 152, The scheme checks on endpoint use case-sensitive strip_prefix and miss mixed-case schemes; update the conversion that sets http_endpoint to do case-insensitive matching: create a lowercase copy (e.g., let lower = endpoint.to_ascii_lowercase()), test lower.strip_prefix("grpc://") and lower.strip_prefix("grpcs://"), and when a match is found compute the original address suffix by slicing the original endpoint past the matched scheme length (so you preserve original casing/host) and then format!("http://{addr}") or format!("https://{addr}") accordingly; update the logic that sets http_endpoint (same variable) and reuse the original endpoint variable for the suffix.crates/grpc_client/src/trtllm_service.rs (1)
144-151:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winImplement case-insensitive scheme matching.
The scheme checks use case-sensitive
strip_prefixand won't handle mixed-case schemes likeGRPC://orGrPcS://. Past review analysis confirms that direct callers likeworker.rs:952pass URLs without normalization, and sincegrpc_base_urlpreserves original URL casing, mixed-case schemes will fail the conversion and cause connection errors.🔧 Proposed fix
// Convert gRPC schemes to tonic-compatible HTTP(S) endpoints. - let http_endpoint = if let Some(addr) = endpoint.strip_prefix("grpc://") { + let lower = endpoint.to_ascii_lowercase(); + let http_endpoint = if let Some(_) = lower.strip_prefix("grpc://") { + let addr = &endpoint[7..]; format!("http://{addr}") - } else if let Some(addr) = endpoint.strip_prefix("grpcs://") { + } else if let Some(_) = lower.strip_prefix("grpcs://") { + let addr = &endpoint[8..]; format!("https://{addr}") } else { endpoint.to_string() };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/trtllm_service.rs` around lines 144 - 151, The current scheme matching uses case-sensitive strip_prefix on endpoint and will miss mixed-case schemes; fix by computing a lowercase copy (e.g. let lower = endpoint.to_ascii_lowercase()) and perform strip_prefix checks against lower (e.g. lower.strip_prefix("grpc://") / lower.strip_prefix("grpcs://")), then map the returned suffix back to the original endpoint string to build http_endpoint so original host casing is preserved; update the logic that sets http_endpoint (using endpoint and the strip_prefix results) to use these case-insensitive checks instead of direct strip_prefix calls.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@crates/grpc_client/src/mlx_engine.rs`:
- Around line 125-131: The scheme-matching for building http_endpoint in the
mlx_engine.rs block (the if/else that currently uses
endpoint.strip_prefix("grpc://") and strip_prefix("grpcs://")) is case-sensitive
and fails for mixed/upper-case schemes; update this block to perform
case-insensitive detection (e.g. compare a lowercased view of endpoint or use a
case-insensitive starts-with) and then strip only the scheme portion while
preserving the original remainder of endpoint so GrpcClient::connect() (and
grpc_base_url in util.rs) receive the correct host/path; adjust the logic around
the http_endpoint variable and the endpoint handling to use these
case-insensitive checks.
In `@crates/grpc_client/src/sglang_scheduler.rs`:
- Around line 145-152: The scheme checks on endpoint use case-sensitive
strip_prefix and miss mixed-case schemes; update the conversion that sets
http_endpoint to do case-insensitive matching: create a lowercase copy (e.g.,
let lower = endpoint.to_ascii_lowercase()), test lower.strip_prefix("grpc://")
and lower.strip_prefix("grpcs://"), and when a match is found compute the
original address suffix by slicing the original endpoint past the matched scheme
length (so you preserve original casing/host) and then format!("http://{addr}")
or format!("https://{addr}") accordingly; update the logic that sets
http_endpoint (same variable) and reuse the original endpoint variable for the
suffix.
In `@crates/grpc_client/src/trtllm_service.rs`:
- Around line 144-151: The current scheme matching uses case-sensitive
strip_prefix on endpoint and will miss mixed-case schemes; fix by computing a
lowercase copy (e.g. let lower = endpoint.to_ascii_lowercase()) and perform
strip_prefix checks against lower (e.g. lower.strip_prefix("grpc://") /
lower.strip_prefix("grpcs://")), then map the returned suffix back to the
original endpoint string to build http_endpoint so original host casing is
preserved; update the logic that sets http_endpoint (using endpoint and the
strip_prefix results) to use these case-insensitive checks instead of direct
strip_prefix calls.
In `@crates/grpc_client/src/vllm_engine.rs`:
- Around line 145-152: The current scheme matching on `endpoint` uses
case-sensitive `strip_prefix` and fails for mixed-case schemes; update the
conversion that sets `http_endpoint` so it matches schemes case-insensitively by
comparing a lowercased view (e.g., `let lower = endpoint.to_lowercase()`) and,
when a match is found, take the remainder from the original `endpoint` by
slicing with the scheme length (so casing of the address is preserved) and then
format to "http://{addr}" or "https://{addr}". Make this change in the block
that computes `http_endpoint` (the code referencing `endpoint` and producing
`http_endpoint`) so both "grpc://" and "grpcs://" are handled regardless of
case.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 8bec4bca-a9c8-456a-9ee7-5f3ca1680480
📒 Files selected for processing (6)
crates/grpc_client/src/mlx_engine.rscrates/grpc_client/src/sglang_scheduler.rscrates/grpc_client/src/trtllm_service.rscrates/grpc_client/src/vllm_engine.rsmodel_gateway/src/workflow/steps/local/detect_connection.rsmodel_gateway/src/workflow/steps/util.rs
c20d500 to
045ee7c
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
model_gateway/src/workflow/steps/local/create_worker.rs (1)
301-313: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winConsider case-insensitive scheme detection for consistency.
The function uses case-sensitive
starts_with()whileutil.rshelpers use case-insensitive scheme parsing. For consistency with RFC 3986 and the rest of the codebase, consider using the same approach asutil.rs.♻️ Proposed refactor
+use super::util::strip_protocol; + fn normalize_url(url: &str, connection_mode: ConnectionMode) -> String { - if url.starts_with("http://") - || url.starts_with("https://") - || url.starts_with("grpc://") - || url.starts_with("grpcs://") - { - url.to_string() + let scheme = url.split_once("://").map(|(s, _)| s.to_ascii_lowercase()); + let has_scheme = matches!( + scheme.as_deref(), + Some("http") | Some("https") | Some("grpc") | Some("grpcs") + ); + if has_scheme { + url.trim_end_matches('/').to_string() } else { match connection_mode { ConnectionMode::Http => format!("http://{url}"), ConnectionMode::Grpc => format!("grpc://{url}"), } } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@model_gateway/src/workflow/steps/local/create_worker.rs` around lines 301 - 313, The URL scheme checks in create_worker.rs are currently case-sensitive (using starts_with) which diverges from util.rs; update the logic that normalizes the worker URL (the block that checks url.starts_with("http://")/ "https://" / "grpc://" / "grpcs://") to perform case-insensitive scheme detection or, better, reuse the existing util.rs helper for scheme parsing (or the Url parsing helper) and then fall back to formatting with ConnectionMode::Http / ConnectionMode::Grpc when no scheme is present; reference the same helper used elsewhere to ensure RFC 3986–compatible, case-insensitive behavior and remove the manual starts_with checks.
♻️ Duplicate comments (5)
crates/grpc_client/src/vllm_engine.rs (1)
146-152:⚠️ Potential issue | 🟠 Major | ⚡ Quick winCase-sensitive scheme check remains unfixed despite pipeline warning.
Pipeline failure warns: "Scheme check used to build
http_endpointinVllmEngineClient::connectis case-sensitive; mixed-case gRPC schemes can be missed and cause incorrect/failed HTTP(S) conversion."The code continues to use case-sensitive
strip_prefix, so URLs likeGRPC://host:portorGRPCS://host:portfrom callers such asworker.rs:952will fail to convert, resulting in connection failures.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/vllm_engine.rs` around lines 146 - 152, The scheme-check in VllmEngineClient::connect uses case-sensitive endpoint.strip_prefix which misses mixed-case schemes (e.g., "GRPC://"); update the logic that builds http_endpoint to do a case-insensitive prefix check on endpoint (e.g., compare a lowercase copy of endpoint) and then extract the original address substring to construct "http://{addr}" or "https://{addr}" so the host/port casing and rest remain intact; modify the code around http_endpoint and the endpoint.strip_prefix usage to use the lowercase check but derive addr from the original endpoint string.crates/grpc_client/src/mlx_engine.rs (1)
125-131:⚠️ Potential issue | 🟠 Major | ⚡ Quick winCase-sensitive scheme matching was flagged in previous reviews but not fixed.
The current implementation still uses case-sensitive
strip_prefix, so schemes likeGRPC://,GRPCS://, orGrPcS://will not match and will cause connection failures. Direct callers (e.g.,worker.rs:952→GrpcClient::connect()→MlxEngineClient::connect()) pass URLs without normalization, making this a real production risk.The fix was previously described in past review comments but has not been applied to the code.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/mlx_engine.rs` around lines 125 - 131, The scheme matching is case-sensitive and should be made case-insensitive: in the MlxEngineClient::connect() (called from GrpcClient::connect()), compute a lowercased view of the endpoint (or use ASCII case-insensitive checks) to detect prefixes like "grpc://" or "grpcs://", but when building the http/https URL use the original endpoint substring (e.g., endpoint[prefix.len()..]) so you preserve casing in the host/path; update the branch that sets http_endpoint to use this case-insensitive prefix detection on the endpoint variable.crates/grpc_client/src/trtllm_service.rs (1)
145-151:⚠️ Potential issue | 🟠 Major | ⚡ Quick winCase-sensitive scheme matching flagged by pipeline and past reviews.
Pipeline failure explicitly warns: "Scheme matching in
TrtllmServiceClient::connectis case-sensitive (e.g., viastrip_prefix("grpc://")/strip_prefix("grpcs://")), so mixed-case gRPC schemes (e.g.,GRPCS://) may not be converted correctly to HTTP(S) endpoints."The current code still uses case-sensitive matching, which will cause connection failures for mixed-case schemes from direct callers like
worker.rs:952.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/trtllm_service.rs` around lines 145 - 151, The scheme matching is case-sensitive in TrtllmServiceClient::connect (the http_endpoint logic uses endpoint.strip_prefix("grpc://") / "grpcs://"), so normalize comparison by using a lowercased view for matching but slice the original endpoint to preserve address casing: create endpoint_lower = endpoint.to_ascii_lowercase() (or similar), use endpoint_lower.starts_with("grpc://") / "grpcs://") to detect the scheme, then compute the addr by slicing the original endpoint after the matched prefix length and format the http/https URL into http_endpoint; replace the current strip_prefix checks with this case-insensitive approach.crates/grpc_client/src/sglang_scheduler.rs (1)
146-152:⚠️ Potential issue | 🟠 Major | ⚡ Quick winCase-sensitive scheme matching remains unfixed.
Despite previous review comments identifying this issue, the code still uses case-sensitive
strip_prefix("grpc://")andstrip_prefix("grpcs://"). Mixed-case schemes likeGRPC://orGRPCS://from direct callers (e.g.,bindings/golang/src/policy.rs:364,worker.rs:952) will fail to convert and cause connection failures.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/grpc_client/src/sglang_scheduler.rs` around lines 146 - 152, The scheme check on endpoint is case-sensitive and misses mixed-case schemes; change the logic that sets http_endpoint (using the endpoint variable) to compare the scheme case-insensitively: compute a lowercase copy like let lower = endpoint.to_ascii_lowercase(), detect prefixes on lower (e.g., lower.strip_prefix("grpc://") / "grpcs://"), and when a prefix matches, slice the original endpoint to preserve host casing (use the prefix length to take the substring from endpoint) before formatting into "http://{addr}" or "https://{addr}". Ensure this replaces the current strip_prefix branches that use endpoint.strip_prefix directly.model_gateway/src/workflow/steps/local/detect_connection.rs (1)
19-27:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winMake explicit scheme detection case-insensitive to match
util.rs.The helper uses case-sensitive
starts_withchecks whileutil.rsparses schemes case-insensitively (url_schemelowercases,strip_schemeuseseq_ignore_ascii_case). A URL likeGRPC://host:30001bypasses the explicit-scheme short-circuit and falls back to parallel probing. RFC 3986 defines schemes as case-insensitive.🔧 Proposed fix
fn explicit_connection_mode(url: &str) -> Option<ConnectionMode> { - if url.starts_with("grpc://") || url.starts_with("grpcs://") { - Some(ConnectionMode::Grpc) - } else if url.starts_with("http://") || url.starts_with("https://") { - Some(ConnectionMode::Http) - } else { - None - } + let scheme = url.split_once("://")?.0.to_ascii_lowercase(); + match scheme.as_str() { + "grpc" | "grpcs" => Some(ConnectionMode::Grpc), + "http" | "https" => Some(ConnectionMode::Http), + _ => None, + } }Also add a test for mixed-case input (e.g.,
GRPC://localhost:30001).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@model_gateway/src/workflow/steps/local/detect_connection.rs` around lines 19 - 27, The explicit_connection_mode function uses case-sensitive starts_with checks so schemes like "GRPC://..." are missed; change it to detect the scheme case-insensitively by comparing the scheme portion with eq_ignore_ascii_case (or by lowercasing a small prefix) and return Some(ConnectionMode::Grpc) for "grpc" / "grpcs" and Some(ConnectionMode::Http) for "http" / "https"; update explicit_connection_mode to check the scheme with eq_ignore_ascii_case (e.g., check the prefix before "://") and add a unit test that passes mixed-case input such as "GRPC://localhost:30001" to verify the behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@model_gateway/src/config/validation.rs`:
- Around line 838-849: The scheme check in validation.rs for worker_url is
case-sensitive and will reject valid URLs like "HTTP://..."; update the check in
the function that returns ConfigError::InvalidValue for field "worker_url" to
perform a case-insensitive scheme comparison (e.g., lowercasing the URL or
extracting and lowercasing the scheme) so that "http://", "https://", "grpc://",
and "grpcs://" are accepted regardless of case; mirror the approach used in
util.rs (normalize scheme to lowercase before starts_with or compare against a
lowercased list) and ensure the error still reports the original url value when
returning ConfigError::InvalidValue.
---
Outside diff comments:
In `@model_gateway/src/workflow/steps/local/create_worker.rs`:
- Around line 301-313: The URL scheme checks in create_worker.rs are currently
case-sensitive (using starts_with) which diverges from util.rs; update the logic
that normalizes the worker URL (the block that checks
url.starts_with("http://")/ "https://" / "grpc://" / "grpcs://") to perform
case-insensitive scheme detection or, better, reuse the existing util.rs helper
for scheme parsing (or the Url parsing helper) and then fall back to formatting
with ConnectionMode::Http / ConnectionMode::Grpc when no scheme is present;
reference the same helper used elsewhere to ensure RFC 3986–compatible,
case-insensitive behavior and remove the manual starts_with checks.
---
Duplicate comments:
In `@crates/grpc_client/src/mlx_engine.rs`:
- Around line 125-131: The scheme matching is case-sensitive and should be made
case-insensitive: in the MlxEngineClient::connect() (called from
GrpcClient::connect()), compute a lowercased view of the endpoint (or use ASCII
case-insensitive checks) to detect prefixes like "grpc://" or "grpcs://", but
when building the http/https URL use the original endpoint substring (e.g.,
endpoint[prefix.len()..]) so you preserve casing in the host/path; update the
branch that sets http_endpoint to use this case-insensitive prefix detection on
the endpoint variable.
In `@crates/grpc_client/src/sglang_scheduler.rs`:
- Around line 146-152: The scheme check on endpoint is case-sensitive and misses
mixed-case schemes; change the logic that sets http_endpoint (using the endpoint
variable) to compare the scheme case-insensitively: compute a lowercase copy
like let lower = endpoint.to_ascii_lowercase(), detect prefixes on lower (e.g.,
lower.strip_prefix("grpc://") / "grpcs://"), and when a prefix matches, slice
the original endpoint to preserve host casing (use the prefix length to take the
substring from endpoint) before formatting into "http://{addr}" or
"https://{addr}". Ensure this replaces the current strip_prefix branches that
use endpoint.strip_prefix directly.
In `@crates/grpc_client/src/trtllm_service.rs`:
- Around line 145-151: The scheme matching is case-sensitive in
TrtllmServiceClient::connect (the http_endpoint logic uses
endpoint.strip_prefix("grpc://") / "grpcs://"), so normalize comparison by using
a lowercased view for matching but slice the original endpoint to preserve
address casing: create endpoint_lower = endpoint.to_ascii_lowercase() (or
similar), use endpoint_lower.starts_with("grpc://") / "grpcs://") to detect the
scheme, then compute the addr by slicing the original endpoint after the matched
prefix length and format the http/https URL into http_endpoint; replace the
current strip_prefix checks with this case-insensitive approach.
In `@crates/grpc_client/src/vllm_engine.rs`:
- Around line 146-152: The scheme-check in VllmEngineClient::connect uses
case-sensitive endpoint.strip_prefix which misses mixed-case schemes (e.g.,
"GRPC://"); update the logic that builds http_endpoint to do a case-insensitive
prefix check on endpoint (e.g., compare a lowercase copy of endpoint) and then
extract the original address substring to construct "http://{addr}" or
"https://{addr}" so the host/port casing and rest remain intact; modify the code
around http_endpoint and the endpoint.strip_prefix usage to use the lowercase
check but derive addr from the original endpoint string.
In `@model_gateway/src/workflow/steps/local/detect_connection.rs`:
- Around line 19-27: The explicit_connection_mode function uses case-sensitive
starts_with checks so schemes like "GRPC://..." are missed; change it to detect
the scheme case-insensitively by comparing the scheme portion with
eq_ignore_ascii_case (or by lowercasing a small prefix) and return
Some(ConnectionMode::Grpc) for "grpc" / "grpcs" and Some(ConnectionMode::Http)
for "http" / "https"; update explicit_connection_mode to check the scheme with
eq_ignore_ascii_case (e.g., check the prefix before "://") and add a unit test
that passes mixed-case input such as "GRPC://localhost:30001" to verify the
behavior.
🪄 Autofix (Beta)
✅ Autofix completed
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1479d80e-481e-4e78-bc84-8e2808309fa2
📒 Files selected for processing (8)
crates/grpc_client/src/mlx_engine.rscrates/grpc_client/src/sglang_scheduler.rscrates/grpc_client/src/trtllm_service.rscrates/grpc_client/src/vllm_engine.rsmodel_gateway/src/config/validation.rsmodel_gateway/src/workflow/steps/local/create_worker.rsmodel_gateway/src/workflow/steps/local/detect_connection.rsmodel_gateway/src/workflow/steps/util.rs
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. Fixes Applied SuccessfullyFixed 1 file(s) based on 1 unresolved review comment. Files modified:
Commit: The changes have been pushed to the Time taken: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 283d6cc131
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@model_gateway/src/config/validation.rs`:
- Around line 838-851: Add unit test coverage to assert the scheme check in
ConfigValidator is case-insensitive by adding a test function (e.g.,
test_validate_case_insensitive_schemes) that constructs RouterConfig with
RoutingMode::Regular { worker_urls: vec![... ] } using mixed-case schemes like
"HTTP://worker:8000", "HTTPS://worker:8000", "GrPc://worker:50051",
"GRPCS://worker:50051" (use PolicyConfig::Random) and assert that
ConfigValidator::validate(&config).is_ok() for each case; this ensures the
to_ascii_lowercase() based scheme validation accepts mixed-case schemes.
🪄 Autofix (Beta)
❌ Autofix failed (check again to retry)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: fb2fb974-90ee-4c04-9f15-40fb1033493c
📒 Files selected for processing (1)
model_gateway/src/config/validation.rs
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. The branch was updated while autofix was in progress. Please try again. |
Signed-off-by: Weiwei Zheng <[email protected]>
Fixed 1 file(s) based on 1 unresolved review comment. Co-authored-by: CodeRabbit <[email protected]>
283d6cc to
90ae9c4
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
model_gateway/src/workflow/steps/local/create_worker.rs (1)
300-313:⚠️ Potential issue | 🟠 Major | ⚡ Quick winHandle existing URL schemes case-insensitively in
normalize_url.Line 301–305 only match lowercase prefixes. A valid mixed-case input can be double-prefixed and become invalid.
🔧 Proposed fix
fn normalize_url(url: &str, connection_mode: ConnectionMode) -> String { - if url.starts_with("http://") - || url.starts_with("https://") - || url.starts_with("grpc://") - || url.starts_with("grpcs://") - { + let has_scheme = matches!( + url.split_once("://") + .map(|(scheme, _)| scheme.to_ascii_lowercase()) + .as_deref(), + Some("http") | Some("https") | Some("grpc") | Some("grpcs") + ); + if has_scheme { url.to_string() } else { match connection_mode { ConnectionMode::Http => format!("http://{url}"), ConnectionMode::Grpc => format!("grpc://{url}"), } } } @@ fn normalize_url_preserves_existing_schemes() { @@ assert_eq!( normalize_url("grpcs://localhost:30001", ConnectionMode::Grpc), "grpcs://localhost:30001" ); + assert_eq!( + normalize_url("GRPC://localhost:30001", ConnectionMode::Grpc), + "GRPC://localhost:30001" + ); }Also applies to: 320-349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@model_gateway/src/workflow/steps/local/create_worker.rs` around lines 300 - 313, The URL normalization in normalize_url currently checks prefixes case-sensitively so inputs like "HTTP://host" get double-prefixed; change the check to be case-insensitive by testing a lowercased view of the URL prefix (e.g., url[..scheme_len].to_lowercase()) or converting url.to_ascii_lowercase() for the starts_with checks, and then only add "http://" or "grpc://" based on ConnectionMode when no scheme is present; update the same case-insensitive logic in the similar normalization code elsewhere (the other normalize block referenced) to prevent double-prefixing for mixed-case schemes.
♻️ Duplicate comments (1)
model_gateway/src/workflow/steps/local/detect_connection.rs (1)
19-27:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMake explicit scheme detection case-insensitive.
Line 20–23 currently use case-sensitive prefix checks. Mixed-case explicit schemes (for example,
GRPC://...) skip the explicit branch and can fall back to HTTP-preferred probing, which breaks the explicit-scheme contract.🔧 Proposed fix
fn explicit_connection_mode(url: &str) -> Option<ConnectionMode> { - if url.starts_with("grpc://") || url.starts_with("grpcs://") { - Some(ConnectionMode::Grpc) - } else if url.starts_with("http://") || url.starts_with("https://") { - Some(ConnectionMode::Http) - } else { - None - } + let scheme = url.split_once("://")?.0.to_ascii_lowercase(); + match scheme.as_str() { + "grpc" | "grpcs" => Some(ConnectionMode::Grpc), + "http" | "https" => Some(ConnectionMode::Http), + _ => None, + } } @@ fn explicit_connection_mode_honors_grpc_scheme() { @@ assert_eq!( explicit_connection_mode("grpcs://localhost:30001"), Some(ConnectionMode::Grpc) ); + assert_eq!( + explicit_connection_mode("GrPc://localhost:30001"), + Some(ConnectionMode::Grpc) + ); }Also applies to: 126-158
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@model_gateway/src/workflow/steps/local/detect_connection.rs` around lines 19 - 27, The explicit_connection_mode function uses case-sensitive starts_with checks so schemes like "GRPC://" are missed; change it to compare a lowercased view of the URL (e.g., let lower = url.to_ascii_lowercase()) and run the prefix checks against that (check "grpc://", "grpcs://", "http://", "https://"), and apply the same case-insensitive prefix normalization to any other explicit-scheme checks elsewhere in this file (the other explicit-scheme detection block) so mixed-case schemes are handled consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@model_gateway/src/workflow/steps/local/create_worker.rs`:
- Around line 300-313: The URL normalization in normalize_url currently checks
prefixes case-sensitively so inputs like "HTTP://host" get double-prefixed;
change the check to be case-insensitive by testing a lowercased view of the URL
prefix (e.g., url[..scheme_len].to_lowercase()) or converting
url.to_ascii_lowercase() for the starts_with checks, and then only add "http://"
or "grpc://" based on ConnectionMode when no scheme is present; update the same
case-insensitive logic in the similar normalization code elsewhere (the other
normalize block referenced) to prevent double-prefixing for mixed-case schemes.
---
Duplicate comments:
In `@model_gateway/src/workflow/steps/local/detect_connection.rs`:
- Around line 19-27: The explicit_connection_mode function uses case-sensitive
starts_with checks so schemes like "GRPC://" are missed; change it to compare a
lowercased view of the URL (e.g., let lower = url.to_ascii_lowercase()) and run
the prefix checks against that (check "grpc://", "grpcs://", "http://",
"https://"), and apply the same case-insensitive prefix normalization to any
other explicit-scheme checks elsewhere in this file (the other explicit-scheme
detection block) so mixed-case schemes are handled consistently.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: e135556c-d8b8-4aed-8b63-b7b5a31c9845
📒 Files selected for processing (8)
crates/grpc_client/src/mlx_engine.rscrates/grpc_client/src/sglang_scheduler.rscrates/grpc_client/src/trtllm_service.rscrates/grpc_client/src/vllm_engine.rsmodel_gateway/src/config/validation.rsmodel_gateway/src/workflow/steps/local/create_worker.rsmodel_gateway/src/workflow/steps/local/detect_connection.rsmodel_gateway/src/workflow/steps/util.rs
Description
Problem
Explicit worker URL schemes were not fully respected during local connection mode detection. A worker configured with
grpc://host:portcould still be probed over HTTP, and if the HTTP health check succeeded, the worker could be classified asConnectionMode::Httpdespite the explicit gRPC scheme.Solution
Honor explicit URL schemes before falling back to auto-detection:
grpc://...runs only the gRPC reachability check and setsConnectionMode::Grpchttp://...andhttps://...run only the HTTP reachability check and setConnectionMode::Httphost:portURLs keep the existing behavior of probing both protocols, with HTTP priority if both passChanges
Test Plan
Reproduced the issue by reviewing
DetectConnectionModeStep: before this change, all URLs were passed into both HTTP and gRPC probes, sogrpc://...could still be selected as HTTP when/healthresponded.After this change, explicit schemes are handled before the auto-detection block, so
grpc://...no longer falls through to HTTP probing.Checklist
cargo +nightly fmtpassescargo clippy --all-targets --all-features -- -D warningspassesSummary by CodeRabbit
New Features
Tests
Documentation/Validation